Coolant and ambient temperature control for chillerless liquid cooled data centers

ABSTRACT

Cooling control methods and systems include measuring a temperature of air provided to one or more nodes by an air-to-liquid heat exchanger; measuring a temperature of at least one component of the one or more nodes and finding a maximum component temperature across all such nodes; comparing the maximum component temperature to a first and second component threshold and comparing the air temperature to a first and second air threshold; and controlling a proportion of coolant flow and a coolant flow rate to the air-to-liquid heat exchanger and the one or more nodes based on the comparisons.

RELATED APPLICATION INFORMATION

This application is related to application Ser. No. TBD, Attorney DocketNo. YOR920120012US1 (163-476), entitled “PROVISIONING COOLING ELEMENTSFOR CHILLERLESS DATA CENTERS”, filed concurrently herewith andincorporated herein by reference.

GOVERNMENT RIGHTS

This invention was made with Government support under Contract No.:DE-EE0002894 (Department of Energy). The government has certain rightsin this invention.

BACKGROUND

1. Technical Field

The present invention relates to data center design and, moreparticularly to energy-efficient cooling systems in large data centers.

2. Description of the Related Art

Data centers are facilities that house numerous computer systemsarranged in the form of electronics racks. Typically, a data centerhouses on the order thousands of electronic racks. Each computer systemin a rack may include one or more processors, memory devices,controllers, power converters and manipulators, and other suchelectronic components. Depending upon the state of operation, a computersystem may dissipate on the order of hundreds of Watts to thousands ofWatts. Therefore, a significant amount of cooling is used to keep theelectronic components within an optimum operating temperature range.Server driven power usage amounts to a significant portion of total USenergy consumption. Liquid cooling solutions, which may includetransferring 100% of the heat dissipated by the rack(s) to water,eliminating the facility air conditioning units, use of building chilledwater to cool the racks, use of energy efficient chillers to providerelatively lower temperature coolants to the rack(s), and many otherliquid cooling solutions, have been proposed as a means to reduce datacenter cooling/total power consumption. However, such solutions are farfrom optimal in their cooling energy efficiency.

Furthermore, many cooling systems are at least partially based on aircooling. Cool air is pumped into servers, cools auxiliary components,and exists as warmer air. A heat exchanger cools the air, whichre-enters the server as cool air. Although liquid-cooled components canbe overcooled, the temperature difference between coolant temperatureentering the air heat exchanger and the air temperature leaving the airheat exchanger can become a limiting factor.

SUMMARY

A cooling control method includes measuring a temperature of airprovided to one or more nodes by an air-to-liquid heat exchanger;measuring a temperature of at least one component of the one or morenodes and finding a maximum component temperature across all such nodes;comparing the maximum component temperature to a first and secondcomponent threshold and comparing the air temperature to a first andsecond air threshold; and controlling a proportion of coolant flow and acoolant flow rate to the air-to-liquid heat exchanger and the one ormore nodes based on said comparisons.

A further cooling control method includes measuring a temperature of airprovided to one or more nodes by an air-to-liquid heat exchanger;measuring a temperature of at least one component of the one or morenodes and finding a maximum component temperature across all such nodes;comparing the maximum component temperature to a first and secondcomponent threshold and comparing the air temperature to a first andsecond air threshold; and controlling a proportion of coolant flow and acoolant flow rate to the air-to-liquid heat exchanger and the one ormore nodes based on said comparisons by adjusting one or more valvesthat control relative flow rate between the air-to-liquid heat exchangerand the one or more nodes.

A cooling system includes one or more nodes, each node having at leastone temperature sensor to monitor a temperature of internal nodecomponents; an air-to-liquid heat exchanger configured to accept aliquid coolant input and to provide cooled air to the one or more nodes;a temperature sensor to monitor a temperature of the air provided by theair-to-liquid heat exchanger; a liquid cooling system configured toprovide liquid coolant to components of the one or more nodes; a valveconfigured to control coolant flow to the air-to-liquid heat exchangerand the liquid cooling system based on the temperature of internal nodecomponents and the temperature of the air provided by the air-to-liquidheat exchanger; and a pump configured to provide liquid coolant to theliquid cooling system and the air-to-liquid heat exchanger, having apump strength that is based on the temperature of internal nodecomponents and the temperature of the air provided by the air-to-liquidheat exchanger.

These and other features and advantages will become apparent from thefollowing detailed description of illustrative embodiments thereof,which is to be read in connection with the accompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

The disclosure will provide details in the following description ofpreferred embodiments with reference to the following figures wherein:

FIG. 1 is a diagram of single-loop and double-loop cooling systems;

FIG. 2 is a diagram of an exemplary intra-rack cooling system accordingto the present principles;

FIG. 3 is a diagram of an intra-server cooling system according to thepresent principles;

FIG. 4 is a diagram of an exemplary intra-rack cooling system accordingto the present principles;

FIG. 5 is a diagram of an exemplary intra-rack cooling system accordingto the present principles;

FIG. 6 is a block/flow diagram of an exemplary method for coolingcontrol according to the present principles; and

FIG. 7 is a diagram of an exemplary intra-rack cooling system accordingto the present principles.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The present principles provide for temperature measurements at variouspoints within a cooling system that combines liquid- and air-basedcooling. This temperature information is used to control coolant flowthrough a rack and through individual servers. By tuning the temperaturedifference between liquid coolant and air flowing through the servers,cooling efficiency can be maximized.

Referring now to the drawings in which like numerals represent the sameor similar elements and initially to FIG. 1, an exemplary data centercooling system 100 is shown. The data center includes a number of racks102, which circulate coolant. Low-temperature coolant 112 enters theracks 102, picks up heat, and leaves the racks 102 as high-temperaturecoolant 114. Although the present invention is described herein withrespect to racks of servers, it is contemplated that any appropriatestructure could be employed. In particular, any clustering, grouping, orother organization of computing devices or structures could be cooledusing the present principles.

FIG. 1 shows a system that has both liquid-to-air heat exchangers 104and liquid-to-liquid heat exchangers (LLHx) 108 In a liquid-to-aircooling arrangement, high-temperature coolant 114 passes directly to anair-side outdoor exchanger 104, for example a set of cooling fins. Anyappropriate type of heat exchange may be used in place of theliquid-to-air exchanger 104, including dry coolers, a building's chilledwater supply, a cooling tower, a wet cooler, a building's heating orheat recovery systems, a geothermal loop, or a combination of multiplekinds. In a liquid-to-liquid cooling arrangement, high-temperaturecoolant 114 passes through a paired cooling coil. A heat exchanger 106has a separate coolant circulation system that also feeds into thepaired cooling coil of LLHx 108. The coolant from the heat exchanger 106reduces the temperature of the high-temperature coolant 114 withoutmixing, before dissipating its heat at heat exchanger 106. The LLHxes108 may be optionally turned off by shutting off the flow of coolantthrough the paired cooling coil. Additionally, multiple LLHxes 108 maybe arranged along a single double-loop line, such that external heatdissipation may be controlled by enabling an appropriate number of heatexchangers 108.

The rate of heat transfer at the rack(s) 102 is predominantly governedby the liquid coolant flow rate through them. At the outdoor heatexchangers 104 and 106, the heat transfer rate is governed by theoutdoor heat exchanger's air-side flow rate and the liquid coolant flowrate through the outdoor heat exchanger 104. The heat transfer rate is anon-linear monotonically increasing function of air-side flow rate andliquid coolant flow rate. For any given heat exchanger design, there isa limit to the air-side flow rate and liquid flow rate. These limits areused to guide the heat exchanger selection so as to meet the maximumcooling requirements (the worst case scenario) by a safe margin. “Worstcase scenario” here refers to the highest ambient air temperature andhighest heat dissipation expected at the rack(s), and in a more generalsense, highest heat dissipation at the data center, occurringsimultaneously. The “worst case scenario” should be rare and might noteven occur over the entire life cycle of the data center.

In some more common situations, an electronic rack 102 might bepartially filled. Moreover, with data center provisioning (for example,powering off servers whose resources are not being used, etc.) beingwidely used to reduce the IT power usage, powered-off servers within arack 102 might also be cooled, even those servers which would notgenerate heat. These situations may result in more cooling powerconsumption than is needed for almost the entire life cycle of datacenter. Hence, liquid cooling distribution hardware and controls basedon physical infrastructure and environmental conditions both inside andoutside the data center, may be used to properly optimize the coolingpower consumption and further reduce the data center energy usage.

Referring now to FIG. 2, a system for managed cooling of servers at anintra-rack level is shown. A plurality of managed servers 204 are shown,each connected to a hardware management console (HMC) 206 by amanagement network 202. The HMC 206 controls workload implementation inthe servers 204 and may include, e.g., one or more hypervisor nodes.Each managed server 204 has a corresponding cooling unit 208, and thecooling units 208 are controlled by a cooling component logic controller212 through a cooling management network 210. Together, the coolingcomponents and controls form cooling system 214. The logic controller212 receives information about outdoor ambient conditions, such astemperature information. Because outdoor temperature is related tocooling efficiency, the logic controller 212 can use that information tocontrol factors such as coolant flow rate.

The present principles reduce cooling power consumption by providingliquid cooling only to the components that require cooling. For example,if a managed server 204 is in off-state, then this status informationcan be fed to the cooling logic controller 212, which would then takesteps to close the coolant flow to that server 204 without affecting thecoolant flow to any other server. To take another example, if themanaged server 204 needs to be powered ON, then this information canalso be fed to the cooling logic controller 212 so that cooling to theserver 204 can be activated. Cooling can furthermore be tuned toparticular levels corresponding to the workload at a server 204, withhigher workloads allocating more cooling. This system applies to theinter-rack level as naturally as to the intra-rack level.

Referring now to FIG. 3, a schematic of an air- and liquid-cooled server300 is shown. In addition to liquid-cooled components, such as CPU coldplates 302 and memory banks 304, many components in a server 300 may beair-cooled. For example, hard drives 308 are frequently air-cooled.Additionally, memory banks 304 may be cooled by a combination of liquid-and air-cooling. Cool coolant liquid 312 enters the server 300 from anexternal cooling system. The coolant 312 enters memory banks 304 and CPUcold plates 302, being warmed in the process and becoming warm coolantliquid 314 to exit the server 300.

An air-to-liquid heat exchanger (ALHx) 310 may be mounted on server 300or on the side of a rack 102 as a sidecar unit and is attached to thecoolant lines 312 and 314. The ALHx may be connected to the coolantlines in either order, taking either warm coolant 314 or cool coolant314 as its input, depending on desired air temperature. Air circulateswithin the server 300 by the fans 306 and is warmed by, e.g., harddrives 308 and memory banks 304. The air exits the server as warm airand is then passed through the ALHx 310, which cools the air beforerecirculating it into server 300. There may be substantial airtemperatures within the server 300, and so multiple ALHxes 310 may beemployed to provide uniform conditions.

As noted above, the ALHx 310 may be connected to coolant lines 312 and314 in either order, taking either cool coolant or warm coolant asinput. In some situations, memory banks 304 may be liquid cooled as wellas air cooled. In this case, part of the heat dissipated by the memorybanks 304 goes into the air, while part of the heat goes into the liquidcoolant. This fraction of heat is dependent on the air and liquidtemperature that the memory banks 304 are exposed to. As such, by havingwarmer air enter the server 300, heat going in to the air from thememory banks 304 may be minimized. This increases the efficiency ofcooling at the rack level. The ALHx 310 may also be connected to thecoolant lines 312 and 314 using valves that allow the coolant flow to bereversed through ALHx 310, taking warm coolant, cool coolant, or acombination of the two, as input as circumstances demand. The coolinginput to the ALHx 310 may be controlled using valves 322.

Liquid cooling at the server level may also be tuned. For example,memory banks 304 may be partially populated and individual CPUs 302 mayhave varying workloads or be shut off entirely. Individual memory slotswithin banks 304 may be selectively cooled according to whether thoseslots are in use, and CPU cold plates 302 may be adjusted or shut offusing valves 318 according to CPU usage. Cooling for entire memory banks304 may be shut off using valves 320. Cooling within the server 300 mayfurther be controlled based on direct measurements of ambienttemperature using, e.g., temperature sensor 316. Temperature sensor maybe used to provide direct feedback to, e.g., ALHx 310 as well as toexternal cooling logic 212, which may in turn tune cooling settingsaccording to desired conditions.

Referring now to FIG. 4, an embodiment of an intra-rack cooling systemis shown. A set of servers 300 are connected in parallel to a coolantinlet plenum 404 and a coolant outlet plenum 406. Inlet plenum 404receives cold input coolant 412 from a pump 401 that draws from outsidethe rack 400. Outlet plenum 406 collects warm coolant 410 from theservers 300, which leaves the rack 400 to be cooled as shown above in,e.g., FIG. 1.

An ALHx side car 402 is connected to the input coolant line 412 and theoutput coolant line 410 by actively controlled three-way valves 408.Valves 408 are used to regulate the flow to the inlet plenum 404 and toside car 402. Because the side car 402 is inside the rack 400, byregulating the flow of coolant to the side car 402, the rack ambient airtemperature leaving the side car 402 and entering servers 300 can becontrolled. This side car 402 could be connected either to the coldcoolant line 412, where coolant would flow through the sidecar 402before entering the inlet plenum 404, or to the warm coolant line 410,where coolant would flow through the sidecar 402 after leaving theoutlet plenum 406. In an alternative embodiment, coolant could be drawnfrom both coolant lines 410 and 412. Connecting the side car 402 to thecold coolant line 412 results in cooler air going through the servers300 and may be useful in situations where air-cooled components, such ashard drives 308, require additional cooling. Connecting the side car 402to the warm coolant line 410 results in warmer air going through servers300, which may be useful in situations where heat going into the air atthe server level should be minimized and heat going into the liquid atthe server level should be maximized.

Referring now to FIG. 5, an alternative embodiment of an intra-rackcooling system 500 is shown. A three-way, flow-directing valve 502 isdisposed between side car 402 and the inlet plenum 404, so that liquidcan be directed to one or the other and a relative flow rate can becontrolled. FIG. 5 also shows a set of temperature measurements that maybe used to monitor and control cooling efficiency. In particular, thetemperature of input coolant 412 is measured as T_(wi), the temperatureof the air output by side car 402 is measured as T_(ai), thetemperatures of important components within servers 300 is measured, andthe maximum of those temperatures is measured as T_(C, max), thetemperature of the coolant output 410 by outlet plenum 406 is measuredas T_(wo2), and the coolant output by the sidecar 402 is measured asT_(wo1).

The cooling system 500 may optionally include two two-way valves 504 and506. These valves offer more precise control of coolant flow to theinlet plenum 404 and the side car 402. Although adjustments to thethree-way valve 502 and the pump 401 can accommodate any balance ofcoolant flow, doing so may be accomplished with fewer steps simply byadjusting one of the two-way valves 504 and 506.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing. Computer program code for carrying out operations foraspects of the present invention may be written in any combination ofone or more programming languages, including an object orientedprogramming language such as Java, Smalltalk, C++ or the like andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The program codemay execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through any type of network, includinga local area network (LAN) or a wide area network (WAN), or theconnection may be made to an external computer (for example, through theInternet using an Internet Service Provider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks. The computer program instructions may also beloaded onto a computer, other programmable data processing apparatus, orother devices to cause a series of operational steps to be performed onthe computer, other programmable apparatus or other devices to produce acomputer implemented process such that the instructions which execute onthe computer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

Referring now to FIG. 6, a block/flow diagram of a method forcontrolling coolant and air temperature is shown. This can beaccomplished by adjusting the position of valve 502, thereby changingthe proportion of coolant flowing through the side car 402 and servers300. Block 602 begins by setting the pump to maximum RPM and opening thevalve to 50%, thereby sending equal coolant to the side car 402 andinlet plenum 404. Next, block 604 checks the monitored temperatures, inparticular obtaining measurements for T_(ai), T_(wo1), T_(wo2), andtaking the maximum component temperature T_(C, max).

Four set temperatures, T_(spec1), T_(spec2), T_(spec3), and T_(spec4),are used for comparison. These values represent, respectively, a hightemperature threshold for server components, a high temperaturethreshold for the temperature of air leaving the side car 402, a lowtemperature threshold for server components, and a low temperaturethreshold for the temperature of air leaving the side car 402. Block 606determines whether the maximum component temperature, T_(C, max),exceeds the high temperature threshold for components and whether themeasured air temperature, T_(ai), exceeds the high temperature thresholdfor air. If so, it is determined that there is not enough coolant flowto the rack 500. Block 608 increases the RPM of a pump 401, therebyincreasing the flow to the rack 500, proportionally increasing flow tothe inlet plenum 404 and the side car 402. Block 608 may also trigger anotification regarding the status of the determined cooling system. Thismay, for example, include a visual message such an error display on ascreen or an indicator light, or an audio message such as a verbalnotification or a beep code. After the pump RPM is increased at block608, block 630 waits t seconds before returning to block 604. The amountof time waited can be any appropriate amount of time, and may be made todepend on the variability of cooling needs. For example, if the serversoperate under a constant workload, then the wait time may be long. Indata centers where workloads vary over time, the wait time may be madeshorter, to allow for rapid adjustments to cooling parameters.

Block 610 checks whether T_(C, max), falls below the high temperaturethreshold for components and whether the measured air temperature,T_(ai), exceeds the high temperature threshold for air. If so, block 612adjusts valve 502 to increase flow to side car 402. Alternatively,two-way valve 504 may be opened, allowing additional coolant flow to theside car 402. The amount of change may be based on the amount of excessin T_(ai) or may be a fixed percentage. Block 612 may also trigger anotification regarding the status of the determined cooling system, inparticular notifying an administrator that there is too little coolantflow to the side car 402.

Block 614 checks whether T_(C, max), exceeds the high temperaturethreshold for components and whether the measured air temperature,T_(ai), falls below the high temperature threshold for air. In thiscase, block 616 may notify a system administrator that there is toolittle coolant flow to the servers 300. Block 616 also increases flow tothe servers 300 by adjusting valve 502. Alternatively, two-way valve 506may be opened, allowing additional coolant flow to the servers 300. Theamount of change may be based on the amount of excess in T_(C, max), ormay be a fixed percentage.

Block 618 determines whether T_(C, max), falls below the low temperaturethreshold for components and whether the measured air temperature,T_(ai), falls below the low temperature threshold for air. In this case,block 620 may notify a system administrator that there is too muchcoolant flow to the servers 300. Block 620 also decreases the RPM ofpump 401, thereby increasing the flow to the rack 500, proportionallyincreasing flow to the inlet plenum 404 and the side car 402.

Block 622 determines whether T_(C, max), exceeds the low temperaturethreshold for components and whether the measured air temperature,T_(ai), falls below the low temperature threshold for air. In this case,block 624 may notify a system administrator that there is too muchcoolant flow to the sidecar 402. Block 624 also adjusts valve 502 todecrease flow to side car 402. Alternatively, two-way valve 504 may beopened, allowing additional coolant flow to the side car 402. The amountof change may be based on the amount of shortfall in T_(ai) or may be afixed percentage.

Block 626 determines whether T_(C, max), falls below the low temperaturethreshold for components and whether the measured air temperature,T_(ai), exceeds the low temperature threshold for air. In this case,block 628 may notify a system administrator that there is too muchcoolant flow to the servers 300. Block 628 also adjusts valve 502 todecrease flow to servers 300. Alternatively, two-way valve 506 may beopened, allowing additional coolant flow to the servers 300. The amountof change may be based on the amount of shortfall in T_(C, max) or maybe a fixed percentage.

After each adjustment, processing goes to block 630, waits for t secondsas described above, and returns processing to block 604 to repeat. Inthis manner, the cooling parameters may be continually adjusted,including the RPM of pump 402 and the state of the three-way valve 502and two-way valves 504 and 506.

In an alternative embodiment, T_(wi), T_(wo1), or T_(wo2) may be usedinstead of T_(C, max). Additionally, T_(wi) and T_(wo2), along with theliquid coolant flow rate through the servers/nodes, can be used tomeasure the heat load going into the liquid coolant at the server/racklevel. Similarly, T_(wi) and T_(wo1), along with the liquid coolant flowrate through the side car air-to-liquid heat exchanger 402 can be usedto measure the heat load going into the air. The percentage of heat loadgoing into the liquid at the server/rack level is related to thetemperature difference between T_(ai) and T_(wi). The greater T_(ai) iswith respect to T_(wi), the greater the heat load transferred to theliquid will be. Under optimal circumstances, T_(ai) will be as muchlarger than T_(wi) as possible. However, there is an upper as well aslower limit to both Tai and Twi. As such, T_(ai) and T_(wi) can becontrolled to obtain a desired heat load distribution to liquid coolantand to air.

Referring now to FIG. 7, an alternative embodiment of an intra-rackcooling system 700 is shown. Side car 402 is arranged in serial with theinlet plenum 404, such that input coolant flow 412 from pump 401 firstpasses through side car 402 and then enters the inlet plenum 404. Abypass two-way valve 702 is configured to provide a bypass path for thecoolant to skip the side car 402. The valve 702 may be adjusted insteps, such that some coolant flows through side car 402 and some doesnot. This valve 702 may be adjusted in a manner similar to thatdiscussed above in FIG. 6, where relative flow may be adjusted in thesame manner, replacing changes to the three-way valve 502 with openingand closing the two-way valve 702.

Having described preferred embodiments of a system and method forcoolant and ambient temperature control for chillerless liquid cooleddata centers (which are intended to be illustrative and not limiting),it is noted that modifications and variations can be made by personsskilled in the art in light of the above teachings. It is therefore tobe understood that changes may be made in the particular embodimentsdisclosed which are within the scope of the invention as outlined by theappended claims. Having thus described aspects of the invention, withthe details and particularity required by the patent laws, what isclaimed and desired protected by Letters Patent is set forth in theappended claims.

What is claimed is:
 1. A cooling control method, comprising: measuring atemperature of air provided to one or more nodes by an air-to-liquidheat exchanger; measuring a temperature of at least one component of theone or more nodes and finding a maximum component temperature across allsuch nodes; comparing the maximum component temperature to a first andsecond component threshold and comparing the air temperature to a firstand second air threshold; and controlling a proportion of coolant flowand a coolant flow rate to the air-to-liquid heat exchanger and the oneor more nodes based on said comparisons.
 2. The cooling control methodof claim 1, further comprising repeating said measuring, comparing, andcontrolling at periodic intervals.
 3. The cooling method of claim 2,wherein said periodic interval is based on historical rates oftemperature change.
 4. The cooling method of claim 1, whereincontrolling a proportion of coolant flow to the air-to-liquid heatexchanger and the one or more nodes comprises controlling a three-wayvalve that divides an input coolant flow between the air-to-liquid heatexchanger and the one or more nodes.
 5. The cooling method of claim 1,wherein the one or more nodes are connected in serial with theair-to-liquid heat exchanger and wherein controlling a proportion ofcoolant flow to the air-to-liquid heat exchanger and the one or morenodes comprises controlling a two-way bypass valve that allows coolantto flow directly to the liquid cooling system.
 6. The cooling method ofclaim 1, wherein controlling a coolant flow rate to the air-to-liquidheat exchanger and the one or more nodes comprises controlling a pumpstrength that determines a rate of input coolant flow.
 7. The coolingmethod of claim 1, wherein controlling a coolant flow rate to theair-to-liquid heat exchanger comprises controlling a two-way valveconfigured to limit a coolant flow to the air-to-liquid heat exchanger.8. The cooling method of claim 1, wherein controlling a coolant flowrate to the one or more nodes exchanger comprises controlling a two-wayvalve configured to limit a coolant flow to the one or more nodes.
 9. Acooling control method, comprising: measuring a temperature of airprovided to one or more nodes by an air-to-liquid heat exchanger;measuring a temperature of at least one component of the one or morenodes and finding a maximum component temperature across all such nodes;comparing the maximum component temperature to a first and secondcomponent threshold and comparing the air temperature to a first andsecond air threshold; and controlling a proportion of coolant flow and acoolant flow rate to the air-to-liquid heat exchanger and the one ormore nodes based on said comparisons by adjusting one or more valvesthat control relative flow rate between the air-to-liquid heat exchangerand the one or more nodes.
 10. The cooling method of claim 9, whereinthe one or more valves include a three-way valve that divides an inputcoolant flow between the air-to-liquid heat exchanger and the one ormore nodes.
 11. The cooling method of claim 9, wherein the one or morenodes are connected in serial with the air-to-liquid heat exchanger andwherein the one or more valves include a two-way bypass valve thatallows coolant to flow directly to the liquid cooling system.
 12. Thecooling method of claim 9, wherein the one or more valves include atwo-way valve configured to limit a coolant flow to the air-to-liquidheat exchanger.
 13. The cooling method of claim 9, wherein the one ormore valves include a two-way valve configured to limit a coolant flowto the one or more nodes.
 14. A cooling system comprising: one or morenodes, each node having at least one temperature sensor to monitor atemperature of internal node components; an air-to-liquid heat exchangerconfigured to accept a liquid coolant input and to provide cooled air tothe one or more nodes; a temperature sensor to monitor a temperature ofthe air provided by the air-to-liquid heat exchanger; a liquid coolingsystem configured to provide liquid coolant to components of the one ormore nodes; a valve configured to control coolant flow to theair-to-liquid heat exchanger and the liquid cooling system based on thetemperature of internal node components and the temperature of the airprovided by the air-to-liquid heat exchanger; and a pump configured toprovide liquid coolant to the liquid cooling system and theair-to-liquid heat exchanger, having a pump strength that is based onthe temperature of internal node components and the temperature of theair provided by the air-to-liquid heat exchanger.
 15. The cooling systemof claim 14, further comprising control logic configured to compare amaximum internal node component temperature to a first and secondcomponent threshold and to compare the air temperature to a first andsecond air threshold.
 16. The cooling system of claim 15, wherein thecontrol logic is further configured to control the valve and the pumpaccording to said comparisons.
 17. The cooling system of claim 14,further comprising a flow rate valve configured to limit a coolant flowto the air-to-liquid heat exchanger based on the temperature of internalnode components and the temperature of the air provided by theair-to-liquid heat exchanger.
 18. The cooling system of claim 14,further comprising a flow rate valve configured to limit a coolant flowto the liquid cooling system based on the temperature of internal nodecomponents and the temperature of the air provided by the air-to-liquidheat exchanger.
 19. The cooling system of claim 14, wherein the valve isa three-way valve that connects the air-to-liquid heat exchanger and theliquid cooling system to the pump in parallel.
 20. The cooling system ofclaim 14, wherein the air-to-liquid heat exchanger and the liquidcooling system are connected serially and the valve is a two-way bypassvalve.
 21. The cooling system of claim 20, wherein the two-way bypassvalve provides a bypass for the air-to-liquid heat exchanger, such thatopening the two-way bypass valve allows coolant to flow directly to theliquid cooling system.